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Abstract 

Naturalistic instructional approaches are used to provide intentional and systematic instruction 
to young children with disabilities during typically occurring activities. Several naturalistic 
instructional approaches have been described in the extant literature, although different terms 
have been used to refer to these approaches (e.g., activity-based intervention, embedded 
instruction). The purpose of this systematic review was to identify, examine, and summarize the 
empirical literature focused on naturalistic instructional approaches for preschool children with 
disabilities when instruction was delivered in classroom settings. Forty-three studies that met 
established inclusion criteria were reviewed and coded using systematic procedures. Studies 
were coded to permit within- and across-approach comparisons as well as with respect to 
quality indicators for study design features and relationships to reported outcomes. Findings 
suggest a need to specify clearly the contextual and procedural components of naturalistic 
instructional approaches to advance understanding about this evidence-based practice and the 
functional relationships between implementation of the approaches and child learning outcomes. 
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Introduction 


Naturalistic instructional (NI) approaches have been developed to support children’s participa- 
tion and learning in inclusive settings (Hemmeter, 2000; Odom, 2000; Pretti-Frontczak & 
Bricker, 2004; VanDerHeyden, Snyder, Smith, Sevin, & Longwell, 2005; Wolery & Hemmeter, 
2011). These approaches have helped early childhood educators support children’s access to and 
participation in the general preschool curriculum, while giving individualized support and 
instruction in the context of typically occurring classroom activities. They typically involve 
embedding (providing learning trials during naturally occurring or motivating activities) and 
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embedded learning opportunities (providing intentional and systematic instructional episodes 
within and across activities based on children’s individualized learning needs and outcomes). 

Providing embedded learning opportunities for children within and across activities and envi- 
ronments is a recommended practice in early intervention/early childhood special education. 
Practitioners embed instruction within and across activities and environments to provide learning 
opportunities relevant for the context (Division for Early Childhood, 2014; Wolery, 2005). 
Embedded learning opportunities promote child engagement and learning during typically occur- 
ring classroom activities (Snyder, Hemmeter, McLean, Sandall, & McLaughlin, 2013). Using NI 
approaches to embed learning opportunities in typically occurring activities has been associated 
with increases in child engagement (e.g., Malmskog & McDonnell, 1999) and skill acquisition 
(e.g., Fox & Hanline, 1993; Grisham-Brown, Schuster, Hemmeter, & Collins, 2000). In addition, 
use of these approaches has been shown to support generalization and maintenance of learned 
skills (e.g., Kaczmarek, Hepting, & Dzubak, 1996; McDonnell, 1996). 

Hart and Risley’s (1975) work on incidental teaching had a primary influence on the develop- 
ment of NI approaches (Horn & Banerjee, 2009). The passage of PL 99-457, which required 
services for young children with disabilities to be delivered in least restrictive or natural environ- 
ments, further supported the development of NI approaches (Kaiser & Trent, 2009). Over time, 
several different terms have been used to describe these approaches in the extant literature, 
including activity-based intervention, individualized curriculum sequencing model, milieu teach- 
ing, enhanced milieu teaching, naturalistic teaching, transition-based teaching, and embedded 
instruction. Each of these naturalistic approaches involves the use of instructional procedures 
that set the occasion for embedded learning opportunities. 

Despite being referred to by different terms, NI approaches could be considered a “practice” as 
defined by the What Works Clearinghouse (2010, p. 4) because they reflect “a named approach to 
promoting children’s development that staff implements in interacting with children and materials 
in their classroom” and they share several common features (Horn & Banerjee, 2009; Rule, Losardo, 
Dinnebeil, Kaiser, & Rowland, 1998). First, the contexts in which instruction occurs are the typi- 
cally occurring activities, routines, and experiences of a child. Second, the content of instruction 
focuses on learning targets or skills needed by the child to meet activity demands or characteristics 
and to participate more fully in typically occurring activities. Third, each intentional and systematic 
teaching episode is typically child-initiated or initiated by an adult based on the child’s focus of 
attention or interest (1.e., following the child’s lead), and a natural or logically planned consequence 
follows the child’s response. Fourth, the adults who implement the instruction are those who inter- 
act regularly with the child. These four features are typically what distinguish the NI approaches 
described above from other instructional approaches described in the literature. 

Although NI approaches share common features, differences in terminology and variations in 
procedural components emphasized in research and practice have made it challenging to identify, 
analyze, and summarize the empirical literature. Given the emphases on identifying and imple- 
menting recommended and evidenced-based practices in early childhood (Cook & Odom, 2013; 
Division for Early Childhood, 2014; Snyder, 2006; What Works Clearinghouse, 2010), a need 
exists to conduct a descriptive systematic review of the literature on NI approaches. 

The purpose of our systematic review was to examine the empirical literature that met study 
inclusion criteria and focused on NI approaches designed to support embedded learning opportu- 
nities for young children with disabilities in typically occurring activities in classroom settings. 
Six research questions guided the review: 


Research Question 1: Across the identified studies, what term was used to characterize 
the NI approach implemented and how frequently was each approach used? 

Research Question 2: What were the conditions under which the NI approach was 
implemented? 
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Research Question 3: What were the attributes of the children and providers involved in 
the studies? 

Research Question 4: Which of the four features of NI were reported and what were the 
key procedural components of the approach? 

Research Question 5: Which experimental design characteristics and quality indicators 
were present in the studies? 

Research Question 6: To what extent was implementation of the NI approach associated 
with children acquiring, generalizing, or maintaining skills? 


Method 


Inclusion and Exclusion Criteria 


For a study to be included in the review, it had to meet five screening criteria. First, the study had 
to be empirically based research focused on implementation of a NI approach. Second, the study 
had to be published in a peer-reviewed journal. Third, the study had to include at least one child 
with a disability who was between the ages of 36 and 60 months at the beginning of the study. 
Fourth, the study had to report outcome data on child learning. Fifth, the instruction reported in 
the study had to be implemented in the context of typically occurring activities, routines, or tran- 
sitions of preschool classrooms. Studies were eligible for inclusion if the instruction was imple- 
mented in either an inclusive or early childhood special education classroom. Studies focused on 
NI approaches conducted in settings other than a preschool classroom were not eligible for inclu- 
sion. We acknowledge the importance of early childhood inclusion and recommended child- 
focused practices that emphasize service provision in natural learning environments (Division 
for Early Childhood/National Association for the Education of Young Children [DEC/NAEYC], 
2009). We chose to include studies conducted in segregated classrooms, given that NI approaches 
have been used in both inclusive and segregated settings. 


Article Search 


The process used to identify potential studies for the review involved four stages. In the first 
stage, an electronic search of articles published between 1980 and 2013 was conducted using 
all the databases in EBSCO Host and Web of Science, and four databases in Wilson Web (i.e., 
Education Full Text, Education Index Retro, ERIC, Social Science Full Text). Combinations 
of the following search terms or truncated search terms using asterisks were used: embed*, 
transition-based, natural*, incidental, activity-based, milieu/enhanced milieu, responsive 
interaction, individualized curriculum sequencing, strateg*, instruction/intervention/ 
teaching, and presc*. 

In the second stage, an ancestral hand search was conducted of the reference lists of all articles 
that met the inclusion criteria from an initial screening. A reference list for a funded grant focused 
on embedded instruction in early learning written by several of the present study co-authors was 
reviewed to verify all eligible studies appearing on the list were identified through the electronic 
or ancestral searches. In the third stage, names of researchers whose intervention studies focused 
on NI approaches in early childhood settings were used as search terms using the search engines 
described previously. The researcher’s last name and first initial were used to conduct the search. 
In the fourth stage, studies identified through the ancestral search, grant reference list, and name 
search not previously identified in the electronic search were searched in EBSCO Host database 
to generate additional indexing or search terms. These terms were used to conduct an additional 
database search using EBSCO Host. Terms included language, teaching, preschool, imitation, 
and disabilit*. 
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Two searches using these procedures were conducted. The first search was conducted in 
October 2009 (to identify articles published between 1980 and 2008) and the second in May 2013 
(to identify articles published after 2008). The first search was conducted in preparation for a 
potential efficacy trial focused on embedded instruction conducted by several of the present study 
authors. The second search was conducted after the potential efficacy trial was completed. 


Initial Screening 


The four-stage process resulted in 1,013 unique articles. The abstract of each unique article iden- 
tified through the search process was screened by two of the present study authors who were 
doctoral students at the time the study was conducted and the first author using a project-devel- 
oped screening form that was created based on the five inclusion and exclusion criteria described 
previously. When an abstract did not contain sufficient information to make an inclusion or 
exclusion decision, the full article was obtained and reviewed. 

The screening processes resulted in 37 articles that met the inclusion criteria. The primary 
reasons for exclusion of many articles were as follows: (a) They did not involve implementation 
of a NI approach, (b) the NI approach was not implemented in a classroom, or (c) the study did 
not include child outcome data. Two secondary coders who were doctoral-level students indepen- 
dently screened 54% of all abstracts (i.e., 549 articles) to ensure accuracy and consistency of the 
screening. Interrater agreement for screening criteria was calculated by dividing the number of 
agreements by the number of agreements plus disagreements and then multiplying by 100. 
Interrater agreement for the screening was 99.6%. 

The 37 articles that met all inclusion criteria are shown with asterisks in the references. Of 
these 37 articles, 5 included multiple case studies or experiments (Fox & Hanline, 1993; Grisham- 
Brown, Pretti-Frontczak, Hawkins, & Winchell, 2009; Halle, Baer, & Spradlin, 1981; Horn, 
Lieber, Li, Sandall, & Schwartz, 2000; Peck, Killen, & Baumgart, 1989). When an article 
included more than | case study or experiment, each case study or experiment had to meet inclu- 
sion criteria to be included. Across these 5 articles, 12 case studies or experiments were reported; 
11 met the inclusion criteria. Thus, the present review was 43 studies from 37 articles. 


Review Procedures 


The article review process involved three steps. First, a coding form to review identified studies 
was developed by the first two authors. Second, coders were trained to use the coding form 
before they began reviewing the studies included in the present review. Third, coders were asked 
to read each study in its entirety and extract information using the coding form. 


Coding form. The coding form included seven sections. We used the first section to record the term 
used to characterize the NI approach and the purpose of the study. The second section was used to 
record information about the attributes of child and adult participants. The next two sections were 
used to record information about the contexts and conditions under which the NI approach was 
implemented (e.g., study setting, types of activities) and the procedural components of the NI 
approach intervention (e.g., content of targeted skills, instructional procedures, dose of interven- 
tion). The fifth section was used to record information with respect to key characteristics of the 
experimental design and quality indicators (e.g., research design, measurement of dependent vari- 
able, treatment fidelity). The quality of single-case design (SCD) studies was evaluated using 
standards derived from the What Works Clearinghouse SCD technical documentation (Kratoch- 
will et al., 2010) and quality indicators for SCD outlined by Horner and colleagues (2005). The 
quality of group experimental designs was evaluated using nine standards derived from essential 
and desirable quality indicators for group experimental and quasi-experimental research studies 
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(Gersten et al., 2005). In the sixth section, information was recorded about child outcomes (i.e., 
skill acquisition, generalization, and maintenance). In the last section of the coding form, informa- 
tion about the definition or key features of the NI approach as described by the study authors was 
recorded, including which of the four features of NI were described as being part of the 
intervention. 


Training the coders. Following the development of the coding form, the first two authors devel- 
oped a manual that included operational definitions and examples for codes. In addition to the 
first two authors, a third doctoral-level coder was trained. Prior to coding, all coders reviewed the 
manual, coded a practice article, discussed agreements and disagreements, and received feedback 
from the first or second author. Each coded at least three additional articles and received feed- 
back on their coding until agreement met or exceeded 80%. 


Interrater agreement for coding. After all coders met the agreement criterion for training, one 
coder coded all studies that met the inclusion criteria (i.e., 43 studies). To evaluate accuracy and 
consistency of primary coding, a secondary coder independently coded 28% of the studies (i.e., 
12 studies) using the coding form. Interrater agreement was calculated by dividing the number of 
agreements for each code by the number of agreements plus disagreements and then multiplying 
by 100. Interrater agreement was 96% (range = 94%-100%). Disagreements between coders 
were resolved through reviewing the articles and discussions with the first author. 


Results 


The 43 studies that met the inclusion criteria were published from 1981 through 2013. Eight stud- 
ies were conducted in the 1980s, 12 studies in the 1990s, 19 studies from 2000 to 2009, and 4 
studies from 2010 to 2013. Forty studies were SCDs; 3 were group experimental designs. 


Terms Used to Refer to NI Approach 


Across the 43 studies, six different terms were used to characterize the NI approach: embedded 
instruction (n = 15), naturalistic teaching (n = 14), milieu teaching (n = 5), transition-based 
teaching (n = 3), activity-based intervention (n = 3), and individualized curriculum sequencing 
model (n = 1). A definition for each approach based on seminal work is shown in Table 1. We 
characterized two studies as using a combined approach because the authors described the inter- 
vention as including naturalistic and direct instruction. 


Purposes and Conditions Under Which NI Was Implemented 


Study purpose. Thirty-nine of 43 studies were designed to investigate child outcomes associated 
with the intervention agents’ use of the NI approach. In the remaining 4 studies, the primary 
purpose was to evaluate fidelity of implementation by the intervention agents, although child 
outcomes were reported (Harjusola-Webb & Robbins, 2012; Mudd & Wolery, 1987; Schepis, 
Ownbey, Parsons, & Reid, 2000; Schepis, Reid, Ownbey, & Parsons, 2001). 

In 6 of 43 studies, a secondary purpose was to compare instructional approaches or instruc- 
tional formats. Three studies compared activity-based intervention with direct instruction 
(Apache, 2005; Botts, Losardo, Tillery, & Werts, 2014; Losardo & Bricker, 1994). One study 
compared milieu teaching with responsive interaction (Yoder et al., 1995) and another compared 
small group or one-to-one massed-trial instructional format with a distributed-trial format 
(Wolery, Doyle, Gast, Ault, & Simpson, 1993). McDonnell (1996) compared instructional trial 
formats: massed practice only versus massed practice and distributed practice. 
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Table I. Definition of Each Naturalistic Instructional Approach From Seminal Work. 


Approach 


Definition 


El (Snyder, Hemmeter, McLean, 
Sandall, & McLaughlin, 2013) 


Approach that emphasizes identifying preferred times and 
activities when intentional and systematic instructional 


procedures for teaching a child’s priority learning targets are 
implemented in typically occurring activities, routines, and 
transitions 
NT (Rule, Losardo, Dinnebeil, 
Kaiser, & Rowland, 1998) 


Approach that emphasizes the delivery of instruction in natural 
settings, utilizes embedded-distributed trials that follow the 
child’s lead, and use of reinforcement indicated by the child’s 
preferences 

Approach that uses naturalistic, conversation-based teaching 
procedures in which the child’s interest in the environment 
is used as a basis for eliciting elaborated child communicative 
responses 

Approach in which a brief instructional trial to elicit a target 
behavior is implemented at the beginning of a transition from 
one activity to another to use time spent in transitions for 
instruction 


MT (Kaiser, 1993; Kaiser, Yoder, & 
Keetz, 1992) 


TBT (Werts, Wolery, Holcombe- 
Ligon, Vassilaros, & Billings, 1992) 


ABI (Bricker & Cripe, 1992; Pretti- 
Frontczak & Bricker, 2004) 


Approach that uses child-directed transactions, embeds 
children’s individual goals and objectives in routine, planned, or 
child-initiated activities and uses logically occurring antecedents 
and consequences to develop functional and generative skills 

ICS (Mulligan, Guess, Holvoet, & 
Brown, 1980) 


Approach that provides multiple opportunities for children 
to respond to natural antecedents and consequences by 
embedding the targeted skills for instruction across a variety of 
activities, materials, settings, instructors, and responses, all of 
which occur in natural environment 


Note. El = embedded instruction; NT = naturalistic teaching; MT = milieu teaching; TBT = transition-based teaching; 
ABI = activity-based intervention/activity-based instruction; ICS = individualized curriculum sequencing model. 


Study settings. Forty-two studies explicitly identified the type of preschool setting in which the NI 
approaches were implemented. In 37 studies, the intervention was implemented in either inclusive 
(non-Head Start), segregated, or Head Start classrooms. In 5 studies, the intervention was imple- 
mented in more than one type of setting. Twenty-six studies were implemented only in one class- 
room, whereas 16 studies were implemented in more than one classroom. Across all studies, NI 
approaches were implemented in 43 inclusive, non—Head Start early childhood settings in 29 stud- 
ies; 18 segregated classrooms in 12 studies; and 12 Head Start classrooms in 5 studies. Wolery et al. 
(1993) reported that the intervention was implemented in a private preschool classroom, but did not 
specify whether the setting was inclusive or segregated. Table 2 shows settings by each NI approach. 


Activities, routines, and transitions. Intervention agents implemented NI approaches within only one 
activity in 18 studies, within two activities in 9 studies, within three or more activities in 11 studies, 
and reportedly during all classroom activities or routines in 5 studies. Across the 43 studies, authors 
reported that instruction was most often delivered in center-time activities (n = 14), free-play activi- 
ties (1 = 13), meal routines (n = 11), and small-group activities (n = 9). Table 2 shows the activities 
in which the authors reported that the instruction was implemented by each NI approach. 


Attributes of Study Participants 


Child participants. Two hundred eleven preschool children with disabilities participated in the 43 
studies. Data reported below are for these 211 children. Three studies included 26 children 
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Table 2. Percentages for Key Study Features by Naturalistic Instructional Approach. 


Naturalistic instructional approach 


El NT MT TBT ABI ICS Com 

Feature (n= 15) (n= 14) (n= 5) (n = 3) (n = 3) (n= 1) (n = 2) 
Setting? 

Inclusive 87 7 40 33 67 0 50 

Self-contained 13 14 100 33 33 100 0 

Head Start | 14 0 0 0 0 100 
Activities® 

Free-play 0 50 80 0 0 0 100 

Center 60 29 0 0 0 0 50 

Small group 27 7 40 0 33 100 0 
Circle 7 14 60 0 33 0 50 

Large group 27 0 0 0 33 100 0 

Other activities¢ 0 14 40 0 67 100 0 
Routines 

Meal times 13 36 40 0 0 100 50 

Other routines? 27 7 0 0 0 100 0 

Transitions 0 0 0 100 0 0 50 


Note. n refers to number of studies by approach. El = embedded instruction; NT = naturalistic teaching; MT = milieu 
teaching; TBT = transition-based teaching; ABI = activity-based instruction; ICS = individualized curriculum sequencing 
model; Com = combined approach. 

aln some studies, the intervention was implemented in more than one type of setting, so total percentages do not sum 
to 100. 

>In some studies, the intervention was implemented in more than one classroom activity, routine, or transition, so 
total percentages do not sum to 100. Activity type was coded as described by the study authors. 

‘Other activities included gross motor, art, table, one-to-one, work area activities, and activities in playground, gym, 
and hallway. 

4Other routines included grooming, bathroom/diapering, arrival, nap, and unspecified morning and natural routines. 


without disabilities (Culatta, Kovarsky, Theadore, Franklin, & Timler, 2003; Grisham-Brown 
et al., 2009; Grisham-Brown, Ridgley, Pretti-Frontczak, Litt, & Nielson, 2006). Demographic 
data and data on child learning outcomes for these 26 children were not included in the present 
review. In addition, data for 4 children from the McDonnell (1996) study were excluded because 
these participants were either enrolled in kindergarten or primary grades at study entry. 

Child gender was reported for 201 of 211 children (135 males, 66 females). The mean ages of 
children with disabilities were reported in 38 studies (= 51.5 months, SD = 7.8), and children’s 
individual ages and age ranges were reported in 39 studies (range = 24-84 months). 

For type of disability, 96 children were characterized as developmentally delayed or at risk for 
developmental delay. Other disability categories reported were speech and language delay 
(n = 48), autism spectrum disorder (n = 36), Down syndrome (” = 12), cerebral palsy (7 = 5), 
multiple disabilities (1 = 5; for example, deaf-blind, intellectual and physical disabilities), other 
disabilities (n = 7; for example, 13-q syndrome, attention deficit and hyperactivity disorder, men- 
tal retardation) and delays-unspecified (” = 2). Table 3 shows information about child gender, 
age, and type of disability by each NI approach. 

For ethnicity, only 5 of 43 studies reported these data for 16 participating children. These 16 
children were White (n = 12), Black (n = 2), and Latino (n = 2). Data on socioeconomic status 
(SES) were reported in only 3 studies for 14 children. Eleven of 14 children were from families 
of low SES; 3 were from families characterized as middle class. 


76 Journal of Early Intervention 37(1) 


Table 3. Percentages of Study Participant Attributes by Naturalistic Instructional Approach. 


Naturalistic instructional approach 


El NT? MT TBT ABI ICS Com 
Attribute (n= 40) (n=64) (n=45) (n=I1) (n=39) (n=2) (n= 10) 
Child gender 
Male 73 58 64 82 56 100 70 
Female 27 27 36 18 44 0 30 
Child mean age at study entry 54.3 49.5 49.9 54.4 53.0 35.5 54.3 
(in months) 
Child disability category 
Developmental delay or at risk 25 13 80 36 87 100 20 
Speech-language delay 18 4l 4 27 13 0 50 
Autism spectrum disorder 28 27 13 0 0 0 20 
Down syndrome 5 14 0 0 0 0 10 
Other° 25 6 2 36 0 0 0 
El NT MT TBT ABI ICS Com 


Intervention agent role 


Teacher 30 45 68 86 0 100 50 
Assistant teacher 7 8 II 0 0 0 0 
Paraprofessional 30 27 21 0 0 0 0 
Graduate student/researcher 17 12 0 0 67 0 0 
Other! 17 8 0 14 33 0 50 


Note. n refers to the total number of children who participated in studies by approach or the total number of 
intervention agents who participated in studies by approach. El = embedded instruction; NT = naturalistic teaching; 
MT = milieu teaching; TBT = transition-based teaching; ABI = activity-based instruction; ICS = individualized 
curriculum sequencing model; Com = combined approach. 

aln Kohler, Strain, Hoyson, and Jamieson’s (1997) NT study, data on gender were not reported for 10 children, thus, 
percentage for gender for NT approach does not sum to 100. 

bMean age was calculated based on the number of children for whom mean age data were reported. 

cOther includes cerebral palsy, multiple disabilities, other disabilities, and delays-unspecified. 

4Other intervention providers include therapists, personal assistants, special education teachers, and undergraduate 
students. 


Nl agents. Across the 43 studies, a total of 122 adults implemented the NI approach. Based on the 
professional role of the adults, as characterized by study authors, NI was implemented by 57 
preschool teachers, 27 teacher aids, 8 assistant teachers, 15 researcher/graduate students, 7 thera- 
pists, 4 personal assistants, 3 undergraduate students, and | special education teacher. Table 3 
shows the intervention agents’ role by each NI approach. 

Researchers reported information regarding the gender of intervention agents in 13 studies, 
age in 8 studies, and ethnicity in | study. For the 13 studies that reported gender, 35 participants 
were female and 2 participants were male. For the 8 studies that reported age, intervention agents 
ranged in age from 20 to 55 years with a mean age of 30 years (SD = 4.8). 

Information about the intervention agents’ level of education was provided in 19 studies. These 
studies included 51 interventionists who had or were working toward a bachelor’s degree or 
higher, 3 with an associate’s degree, 5 who either held or were working toward a high school 
diploma or equivalent, and 1 with a Child Development Associate credential. In 18 studies, 
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researchers reported information about the teaching experiences of intervention agents. Teaching 
experience ranged from no experience to 22 years with a mean experience of 7.7 years (SD = 5.7). 


Features and Components of NI Approach 


Overview of the four features. Each study was analyzed to determine whether the authors reported 
implementing the four features of NI approaches described by Rule et al. (1998). All 43 studies 
were implemented in classroom environments during ongoing activities (consistent with inclu- 
sion criteria) and addressed skills that appeared to support children’s participation. Although 
following the child’s lead is a distinguishing feature of NI approaches, only 25 of 43 studies 
explicitly stated that instruction was implemented by following the child’s lead. In 17 studies, the 
authors explicitly described that naturally occurring or logically planned consequences followed 
the child’s behavior. In 35 studies, intervention agents were those who interacted regularly with 
participating children. Overall, in 10 studies, the implementation of the NI approach was reported 
with sufficient detail to conclude the intervention included the four features described above. 
Four of these 10 studies were characterized by the authors as naturalistic teaching, 4 as milieu 
teaching, and 2 as embedded instruction. Although only 10 studies explicitly described that all 
four features of NI approaches were implemented, we chose to summarize findings from all stud- 
ies that met study inclusion criteria because we did not know whether these four features were 
not implemented or just not described. Tables 4 and 5 show these 10 studies along with all other 
studies reviewed. 


Skills targeted for instruction. We used two different categorization systems to group skills targeted 
for instruction: by developmental domains or by type of skill outcomes. For developmental 
domain categorization, we grouped targeted skills under one of six categories: preacademic, 
social, communication, motor, adaptive, and cognition. Across 42 studies in which researchers 
reported the targeted skills, 26 addressed one or more target skills from communication, 17 
addressed preacademic skills, 8 addressed social skills, 8 addressed motor skills, 4 addressed 
adaptive skills, and 1 addressed a cognitive skill. 

For type of skill outcome, instructional targets were grouped under four categories: disposi- 
tions (e.g., persistence), chains of behaviors (e.g., washing hands), discrete responses (e.g., nam- 
ing colors), and response classes (e.g., imitating peers). Across the 43 studies, researchers 
investigated the influence of NI approaches on discrete behaviors in 26 studies, response classes 
in 14 studies, and chained behaviors in 13 studies. Dispositions were not investigated in any 
study. Table 6 shows categories addressed by each NI approach. 


Target skill selection. The procedures used to select target skills were reported in 42 studies. In 
some studies, researchers used more than one strategy to identify target skills. Skills were selected 
from participating children’s individualized educational programs in 16 studies, based on assess- 
ment results in 9 studies, based on teacher interview in 7 studies, and based on parent interview 
in 3 studies. In 14 studies, researchers reported that target skills were selected based on their 
presumed importance for the preschool children participating in each study. In 7 studies, research- 
ers reported that the skills selected were those that were not in participating children’s repertoires 
(i.e., had not been acquired). 


Instructional procedures. Forty-two of the 43 reviewed studies reported the instructional proce- 
dures used as part of the NI approach. Based on the description of the procedures provided, as 
shown in Table 7, we organized them under three major categories: planned environmental 
arrangements, adult- or peer-mediated instructional procedures, or other. Planned environmental 
arrangements involve “engineering” the learning environment or modifying learning activities to 
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Table 6. Percentages of Target Skills Addressed From Developmental Domain or Type of Skill by 
Approach. 


Naturalistic instructional approach 


El NT MT TBT ABI ICS Com 


(n=15)  (n=14) (n=5)  (n=3))— (n=3)—s (n=1)—s (n= 2) 


Developmental domain 


Preacademic 47 36 0 100 0 100 50 
Social 13 36 0 0 0 0 50 
Communication 27 7\| 100 67 67 100 100 
Motor 20 21 0 0 33 100 0 
Adaptive 13 14 0 0 0 0 0 
Cognitive 7 0 0 0 0 0 0 
Type of skill 
Disposition 0 0 0 0 0 0 0 
Behavior chain 53 29 0 0 33 0 0 
Discrete response 60 57 60 100 33 100 50 
Response class 33 36 40 0 33 0 50 


Note. n refers to number of studies by approach. Some studies addressed target skills from more than one category. 
Total percentages, therefore, might not sum to 100. El = embedded instruction; NT = naturalistic teaching; 

MT = milieu teaching; TBT = transition-based teaching; ABI = activity-based instruction; ICS = individualized 
curriculum sequencing model; Com = combined approach. 


set the occasion for an embedded learning trial during a typically occurring activity (e.g., placing 
a cup out of reach during snack routine to set the occasion for a child to use a single-word utter- 
ance [e.g., “cup”’] to gain access to the cup). Adult- or peer-mediated instructional procedures are 
those that specify a precise instructional sequence that should be followed for the embedded 
learning trial and typically include a prompt or prompt-fading strategy, a target behavior or 
approximation, and the provision of explicit feedback or consequence (e.g., mand-model, inci- 
dental teaching, system-of-least prompts, constant time delay). The “other” category included 
procedures that did not meet the operational definitions for the previous two categories (e.g., 
visual support). Across 43 studies, one or more planned environmental arrangement procedures 
were used in 16 studies, one or more adult- or peer-mediated instructional procedures were used 
in 39 studies, and one other strategy was used in 2 studies. Table 7 shows the three categories and 
details the instructional procedures used for the adult- or peer-mediated category in the reviewed 
studies by NI approach. 


Dose. Thirty-three studies provided information about the dose of intervention. The number of 
instructional trials or opportunities delivered to a participating child in an intervention session 
was reported in 21 studies and ranged from 3 to 20 trials per target skill (= 7.2). The duration 
of an intervention session was reported in 18 studies and ranged from 5 min to 180 min (M@ = 
23.3). When a study reported a range for the duration of intervention sessions, the lower number 
was used to calculate the mean duration of intervention sessions across studies. The number of 
intervention sessions conducted in a day was reported in 13 studies and ranged from | to 3 ses- 
sions (M = 1.5 sessions/day). In 5 studies, the total duration was reported for the intervention or 
intervention phase and ranged from 6 to 30 weeks (M = 13). 

For the 21 studies that reported the number of instruction trials provided in each intervention 
session, the number of trials or opportunities provided varied across instructional approaches. The 
mean number of instructional trials per target skill in each session was 9.2 in the 5 naturalistic 
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Table 7. Percentages of Studies That Reported Implementing an Instructional Procedure by Approach. 


Naturalistic instructional approach 


EI NT MT TBT ABI ICS Com 
Instructional procedure (n= 15) (n= 14) (n=5) (n=3) (n=28) (n=1) (n=2) 
Planned environmental arrangements 
Environmental arrangements 47 29 60 0 50 0 50 
Curricular modifications 40 0 0 0 0 0 50 
Adult- or peer-mediated? instructional procedures 
Incidental teaching 0 7 40 0 0 0 0 
Elaboration or expansion 0 14 20 0 0 0 0 
Mand or models 40 29 80 0 0 0 50 
Constant time delay 40 0 0 67 50 0 0 
Progressive time delay 7 0 0 33 0 0 0 
Time delay (unspecified) 7 29 60 0 0 0 0 
System-of-least prompts 20 21 0 0 0 100 0 
Most to least prompts 7 0 20 0 0 0 0 
Simultaneous prompting 7 0 0 0 0 0 0 
Prompts (unspecified) 33 29 20 0 0 0 0 
Feedback or consequence strategies 27 29 20 0 0 0 0 
Antecedent strategies 0 0 0 0 100 0 0 
Other 74 0 0 0 0 0 50° 


Note. n refers to total number of studies for each approach. More than one instructional procedure could be used 
in a study, so total percentages by approach do not sum to 100. El = embedded instruction; NT = naturalistic 
teaching; MT = milieu teaching; TBT = transition-based teaching; ABI = activity-based instruction; ICS = individualized 
curriculum sequencing model; Com = combined approach. 

@One activity-based intervention study (Apache, 2005) did not provide specific information about instructional 
procedures used with children. Percentages reported in the table are based on two of three ABI studies in which 
information about instructional procedures was provided. 

No studies implemented peer-mediated instructional procedures; one study encouraged peer interaction during 
teaching episodes but is not reflected in the listed procedures. 

‘Mand or Model refers to mand-model, mand only, or model only. 

4Other El procedures refer to visual supports. 

eOther Com procedures refer to discrete trial. 


teaching studies, 6 in the 10 embedded instruction studies, 3.7 in the 3 transition-based teaching 
studies, 3 in the 1 milieu teaching study, and 12 in the individualized curriculum sequencing 
model study. In an activity-based intervention study (Losardo & Bricker, 1994), 20 instructional 
opportunities were presented to children in each intervention session. 

The mean length of intervention sessions varied across each NI approach. The mean length of 
intervention sessions was 65 min (range = 5-180 min) for three embedded instruction studies, 
21.7 min for three activity-based intervention studies (range = 15-30 min), 12.5 min (range = 
10-20 min) for eight naturalistic teaching studies, and 10 min (range = 5-15 min) for three milieu 
teaching studies. 


Experimental Design Characteristics and Quality Indicators 


Research methods and designs. The designs used in the 40 SCD studies were multiple baseline 
across participants (n = 22), behaviors (n = 11), and settings (7 = 1); alternating treatments (n = 3); 
and AB (n = 2). One study used both a withdrawal and a multiple baseline across participants 
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design (Wolery, Anthony, & Heckathorn, 1998). Table 4 shows experimental designs employed 
in each SCD study by NI approach. The three group experimental studies were two-group pre- 
test/posttest (n = 2; Apache, 2005; Yoder et al., 1995) and a quasi-experimental crossover design 
(Culatta et al., 2003). 


Quality indicators for SCD studies. Table 4 shows experimental design features and quality indica- 
tors present in the 40 SCD studies. In the table, numeric and letter combinations for designs refer 
to categories of different SCDs. Data for each quality indicator in Table 4 are shown by a check 
mark or a numeric value of 0, 1, 2, or 3. The check mark indicates that the dimension of quality 
for the indicator was met. The numeric value refers to a continuum of quality for the indicator 
(i.e., higher number associated with higher quality for the indicator). Information about each 
quality indicator evaluated is provided in the table notes. Below, we summarize information for 
each quality indicator across the reviewed studies. 


Sufficient description of participants. Two of 40 studies reported sufficient information (i.e., gen- 
der, age, functioning, disability category, ethnicity, and SES) to directly replicate the studies with 
similar participants. Thirty provided information about gender, age, and functioning of child 
participants. In 8 studies, demographic information reported for children was very limited. 

For adult participants (i.e., intervention agents), one study reported gender, age, ethnicity, 
role, level of education, and teaching experience, and five studies reported gender, age, role, and 
level of education. Thirty-four studies provided limited information about intervention agents. 


Manipulation of independent variable and measurement of treatment fidelity. Across all 40 SCD 
studies, the independent variable(s) was systematically manipulated with the researcher(s) deter- 
mining how to implement phase or condition changes. Treatment fidelity data were reported in 
25 of 40 studies, and mean treatment fidelity was 80% or greater in 24 of these studies. Only 9 of 
25 studies reported that treatment fidelity data were collected for at least 33% of the intervention 
sessions. 

Treatment fidelity data were collected by direct observation using a procedural checklist in 25 
studies and reported as the percentage of procedures implemented. Of 25 studies in which a 
direct observation method was used to collect treatment fidelity data, 23 reported the mean treat- 
ment fidelity scores. Across these 23 studies, mean treatment fidelity was 93% (range of means = 
70%-100%) during the intervention phase of the study. 

We analyzed further the 25 studies that used a direct observation method to collect fidelity 
data by reading the description of the fidelity checklist used in the study. Each study was coded 
as belonging to one of three categories: (a) description of fidelity checklist items included proce- 
dural components associated with an instructional trial (e.g., “present stimulus and attending cue, 
secure attention, present the task direction; wait the appropriate interval, provide the model, 
provide appropriate consequence, provide correct inter-trial interval’’), (b) description of fidelity 
checklist items included “generic” statements (e.g., “fidelity measure included the major compo- 
nents and features of activity-based intervention and direct instruction”), or (c) description of 
fidelity checklist items was not provided. Across these 25 studies, 16 used a fidelity checklist 
including procedural components associated with an instructional trial, 5 used generic statements 
to describe fidelity checklist items, and 5 studies reported treatment fidelity data but did not pro- 
vide information about the fidelity checklist items. 


Measurement of dependent variable. The dependent variable(s) was repeatedly measured across 
all 40 studies by more than one observer. Thirty-nine used behavioral observations and 1 used 
both behavioral observations and standardized assessments (Macy & Bricker, 2007) to measure 
child outcomes. In the studies in which behavioral observations were used to evaluate children’s 
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performance, researchers conducted live observation in 34 studies and videotaped the sessions in 
6 studies. Across these 40 studies, 14 reported the length of the dependent measures in data col- 
lection session, which ranged from 5 to 30 min with an overall mean of 13.9 min. When behav- 
ioral observations were used as dependent measures, researchers reported data using various 
metrics. These included reporting percentage of correct responses (n = 23), the frequency with 
which a target behavior occurred (” = 8), the rate at which the target behavior occurred (n = 4), 
the percentage of intervals in which a target behavior occurred (n = 4), and rating of level of child 
independence (n = 1). 


Interobserver agreement (IOA). IOA data were reported in each study, and the reported mean IOA 
was 80% or greater across all studies. Of 40 studies, the IOA data were reported for at least 20% 
of sessions across condition in 32 studies. In 8 studies, the IOA was reported for less than 20% of 
the sessions across conditions. 

IOA was reported using percentage agreement in 39 studies and a generalizability coefficient 
in | study (McCathren, 2000). Across the 39 studies that used percentage agreement to report 
IOA, 36 reported the mean IOA on child behaviors. Mean IOA ranged from 80% to 100% with 
an overall mean of 95% during the intervention phase. 


Demonstration of experimental effect. Thirty-three of 40 SCD studies included at least three 
attempts to demonstrate an experimental effect on child learning outcomes at three different 
points in time or with three different phase repetitions. For the number of data points in each 
study phase (excluding generalization or maintenance), each phase included 5 or more data 
points in 12 studies, 3 or 4 data points in 25 studies, and fewer than 3 data points in 2 studies. 


Quality indicators for group experimental design studies. Table 5 shows the quality indicators for 
each group experimental design study by approach. Overall, none of the studies met all nine 
standards. Two studies met five of nine standards (Culatta et al., 2003; Yoder et al., 1995), and 
one study met four of nine standards (Apache, 2005). Three standards, adequate description of 
treatment and comparison conditions, reporting reliability for outcome measures, and reporting 
effect sizes or adequate information to calculate effect sizes, were met by all three studies. Stan- 
dards with respect to comparable groups and treatment fidelity were met by two studies (Culatta 
et al., 2003; Yoder et al., 1995). Standards regarding random assignment, use of multiple out- 
come measures, and providing evidence for validity of outcome measures were each met by one 
study. Attrition was not reported in any of the three studies. 


Relationships Between Intervention and Child Outcomes 


We coded whether authors reported implementation of the NI approach was functionally related 
to skill acquisition, generalization, or maintenance. Table 8 shows the percentages of studies 
reporting acquisition, generalization, and maintenance data for children and the percentage of 
children who acquired, generalized, or maintained target skills by each approach. 


Skill acquisition. Data on skill acquisition were reported for all 211 preschool children with dis- 
abilities in 43 studies. Only 5 of 43 studies reported skill acquisition data in relation to a preiden- 
tified criterion. Positive skill acquisition outcomes associated with the use of NI approaches were 
reported for a total of 207 of the 211 young children with disabilities. Based on the visual analysis 
of data reported in the studies or descriptions provided by the authors, 189 had improved perfor- 
mance during or after intervention over their baseline performance (e.g., increase in number/ 
percentage of correct response) and 16 children acquired targeted skills as measured by reaching 
a specified criterion (i.e., predetermined level of performance; for example, 83% correct 
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Table 8. Percentages of Child Outcomes by Approach. 


Naturalistic instructional approach 


El NT MT TBT ABI ICS Com 


(n=15)  (n=14)  (n=5)— (n=3)—s (n=3)—s (n=) ~— (n= 2 


Studies reporting 


Skill acquisition 100 100 100 100 100 100 100 

Generalization 40 36 40 67 67 100 0 

Maintenance 40 50 40 67 67 100 0 
El NT MT TBT ABI ICS Com 


(n=40)  (n=64) (n=45) (n=I1)— (n=39) s(n =2)_~— (n= 10) 


Results? 
Skill acquisition 100/98 100/100 100/100 100/100 100/100 100/100 100/70 
Generalization 35/30 17/16 LI/11 64/64 28/28 100/100 0 
Maintenance 38/35 38/33 LI/11 36/27 28/28 100/100 0 


Note. n for studies reporting skill acquisition, generalization, and maintenance refers to total number of studies by 

approach. n for results refers to total number of children who participated in studies by approach. El = embedded 
instruction; NT = naturalistic teaching; MT = milieu teaching; TBT = transition-based teaching; ABI = activity-based 
instruction; ICS = individualized curriculum sequencing model; Com = combined approach. 

aResults shown as % of children for whom data reported per percentage of children with positive outcomes. 


responding for two consecutive intervention sessions). Two children did not reach criteria (i.e., 
80% or 100% accuracy for 2 or 3 days across 2 studies) but demonstrated increases in their per- 
centages of correct responses. 

Four children with disabilities did not demonstrate acquisition of targeted skills in relation to 
an intervention provider’s use of a NI approach. In the Culatta et al. (2003) study, three children 
with disabilities did not demonstrate the target skills independently after instruction was pro- 
vided using naturalistic teaching. In the Grisham-Brown et al. (2000) study, one participating 
child’s percentage of correct responses did not increase above baseline levels after embedded 
instruction was initiated. 


Generalization. Generalization data were reported for 50 children in 18 studies. These included 
generalization data across settings (n = 15), materials (n = 4), people (n = 3), and responses 
(n = 1). Across these 18 studies, authors reported some type of evidence that 47 of these children 
generalized the skills they learned. 


Maintenance. Maintenance data were reported for 61 participating children across 20 studies. Of 
the studies reporting maintenance data, 9 provided information about when maintenance data 
were collected. On average, the last maintenance probe was conducted 3.1 weeks after the inter- 
vention ended or when the target children reached acquisition criterion (range = 1-20 weeks). 
Fifty-six of the 61 children demonstrated targeted skills during maintenance probe sessions. 


Discussion 


The purposes of this systematic review were to characterize descriptively the empirical literature 
focused on NI approaches on several important dimensions relevant for implementing these 
approaches and to examine the extent to which quality indicators for single-case experimental 
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and group experimental designs were reflected in the reviewed studies. A total of 37 articles that 
included 43 studies met the inclusion criteria and were analyzed using an investigator-developed 
coding system. In the following sections, we discuss findings of the present study with respect to 
each research question, present delimitations and limitations of the present review, and offer 
implications for future research and practice. 


Terms Used to Characterize NI Approaches 


Six named NI approaches were identified in the empirical literature: (a) naturalistic teaching, (b) 
embedded instruction, (c) activity-based intervention, (d) milieu teaching, (e) transition-based 
teaching, and (f) individualized curriculum sequencing model. The most common term used in 
the reviewed studies was embedded instruction, followed by naturalistic teaching. The findings 
related to the terms used most often to refer to the approach were not unexpected, given the 
explicit emphasis in these approaches on embedding instruction during typically occurring activ- 
ities using naturalistic teaching strategies. 


Conditions Under Which Studies Were Conducted 


The majority of studies were conducted in inclusive settings, consistent with recommendations 
that NI approaches should be used to support the access and participation of young children with 
disabilities in these settings (DEC/NAEYC, 2009). Across the reviewed studies, instruction was 
embedded in a variety of child-initiated and adult-directed activities. From the information pro- 
vided in the reviewed studies about how learning targets were selected, it was difficult to deter- 
mine whether targets were designed to promote situated learning opportunities and functional 
skills children would need to participate more fully in typically occurring activities (Dunst et al., 
2001) or to promote general skill development. 

Few studies described identifying a logical “fit” between the content of the learning target and 
the activity, routine, or transition in which instruction was embedded. Contemporary conceptual- 
izations suggest that it is important to assess child, environment, and activity “fit” when making 
decisions about what to teach and when to teach when using NI approaches (Snyder et al., 2013). 
Future studies should provide additional specificity about the conditions under which NI 
approaches are implemented, particularly with respect to why a particular skill is taught in a 
designated activity or activities. This would further advance knowledge about the conditions 
under which NI approaches are likely to be effective by elucidating whether naturally occurring 
motivational variables for the target behavior are present. 


Attributes of Study Participants 


Studies included in the present review involved young children with a variety of disabilities and 
79% of the intervention agents were adults who regularly interacted with participating children. 
These findings suggest that the research on NI approaches is distinguished by its frequent imple- 
mentation in typical practice settings by “authentic” providers with children from the population 
of preschool children with disabilities. 

Detailed information about the attributes of study participants was not provided in the major- 
ity of studies. Goldstein (2002) reported similar findings related to limitations in the reporting of 
participant attribute data in his review of 60 studies focused on communication interventions for 
children with autism. He and others (Gersten et al., 2005; Horner et al., 2005) have emphasized 
the need for developing a set of conventions to standardize the sharing of descriptive information 
about study participants, including both child participants and individuals who implement the 
intervention. Future research should include sufficient descriptions of adult and child participants 
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(e.g., ethnicity, English language learner, age, gender, level of education, years of experience, 
functional abilities) to identify the characteristics of intervention agents who can implement NI 
approaches with fidelity and the attributes of children for whom the NI approaches are 
effective. 


Four Features of NI 


Across the reviewed studies, four features of NI approaches highlighted in the extant literature 
were not always explicitly described, although there was some evidence that these features were 
present. Only 23% of the studies explicitly described implementation of all four features of NI. 
Because various terms are used to refer to NI approaches, explicitly describing how the four 
“naturalistic” features are implemented will be important for advancing the knowledge base 
about this evidence-based practice, particularly when these approaches are compared with other 
instructional approaches that do not have most or all of these features. 


Key Procedural Components 


We characterized procedural components in the present study by examining what content was 
emphasized in a naturalistic teaching episode, where teaching episodes occurred, and which 
instructional procedures were used during teaching episodes. Specifying and operationally defin- 
ing the procedural components of the NI approach is important for at least two reasons. First, it 
facilitates systematic or direct replication. Second, it permits comparative analyses about compo- 
nents that are or are not delivered across approaches with similar or distinct features. Comparative 
analyses facilitate the identification of key or active ingredients and the examination of relative 
efficacy or effectiveness of components in relation to child learning outcomes (Hulleman & 
Cordray, 2009). 

With respect to content emphasized, we found that learning targets were largely focused on 
communication or preacademic skills. Moreover, the majority of skills included in learning tar- 
gets were discrete skills or focused on chained behaviors. As Wolery and Hemmeter (2011) 
noted, additional research is needed to determine which types of skills are most efficiently and 
effectively taught using NI approaches, particularly given a child’s phase and pace of learning 
and the characteristics of the activities in which the child participates. 

In the majority of the reviewed studies, teaching episodes occurred primarily in one or two 
classroom activities, routines, or transitions. In addition, there was variation in where teaching 
episodes occurred depending on the NI approach that was implemented. For example, teaching 
episodes in the majority of embedded instruction, naturalistic teaching, and milieu teaching stud- 
ies were implemented during free-play or center-time activities, while teaching episodes occurred 
only during transitions between activities (as would be expected) in transition-based teaching. 
The findings related to embedded instruction, naturalistic teaching, and milieu teaching were 
somewhat unexpected, given the emphasis in NI approaches on providing embedded learning 
opportunities across a variety of activities. Perhaps this finding can be explained by challenges 
associated with collecting data in preschool classrooms in ways that reflect the complex nature 
of where teaching episodes occur in relation to the classroom schedule. Documenting teaching 
episodes occurring across numerous activities would require significant resources for data col- 
lection and analysis, which has likely led most researchers to identify one or two activities in 
which to evaluate implementation of teaching episodes. 

Even descriptive studies that have quantified where preschool teachers have provided embed- 
ded teaching episodes on priority learning targets have limited observations to an average of 86 
min (Pretti-Frontczak & Bricker, 2001) or 20 min (Noh, Allen, & Squires, 2009), which does not 
capture an entire preschool day and might not permit observations during different types of 
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activities (e.g., child-initiated, teacher-directed, routines, transitions). Future research might 
focus on systematically collecting data across various categories of activities that occur during 
the preschool day, characterizing more precisely when teaching episodes occur during these 
activities, and examining child learning outcomes in relation to the types of activities in which 
instruction occurred. 

When examining the instructional procedures used to create embedded learning opportunities, 
we identified two major categories of procedures: (a) environmental arrangements and (b) adult- 
or peer-mediated systematic instructional procedures (e.g., constant time delay, system-of-least 
prompts). Instructional procedures were reportedly selected based on their appropriateness for 
teaching the skills targeted for intervention. For example, when motor or adaptive skills were 
targeted, the instructional procedure used typically was the system-of-least prompts. When lan- 
guage skills were addressed, environmental arrangements, mand, or modeling procedures were 
used, or were used in combination with one another. In 18 studies, some of the instructional 
procedures used were not described sufficiently to permit an understanding of the components of 
embedded learning trials (i.e., the nature and relationships among the antecedents, behaviors, and 
consequences). To advance knowledge about intentional and systematic learning trials, research- 
ers should name the instructional procedures used in future studies with particular attention to the 
logical antecedents, consequences, and error corrections (when appropriate) used in these proce- 
dures, which result in complete learning trials for children (VanDerHeyden et al., 2005). 

Intensity of treatment is related to the dose of intervention implemented. The features of dose 
(i.e., rate, length, and distribution), dose frequency (i.e., number of times an intervention is pro- 
vided per day or per week), and total duration of the intervention or intervention phase were 
represented in less than half the studies. Treatment intensity varied widely across the studies in 
which it was reported. Given the limited information reported in the reviewed studies, it was not 
possible to calculate mean cumulative intervention intensity either for all 43 studies or for studies 
associated with each NI approach. To advance reporting practices, researchers might consider 
specifying cumulative intervention intensity and dose form using the definitions provided by 
Warren, Fey, and Yoder (2007). In addition, these authors have advocated for the conduct of 
research focused on promising intervention approaches (e.g., NI) that examine relative efficacy 
and effectiveness under varied treatment intensities. Although 6 studies in the present review 
compared different instructional approaches, only | study focused explicitly on examining select 
aspects of differential treatment intensity specified by Warren etal (McDonnell, 1996). 
McDonnell compared NI for three communication targets per child providing the same cumula- 
tive intervention intensity for each participant but varying the dose of instructional episodes by 
comparing 30 massed trials per session with 15 massed and 15 distributed trials per session. 
Findings showed minimal child-specific differences in the relative effectiveness of the two dose 
formats. With the exception of the McDonnell study, our overall finding about treatment intensity 
is consistent with the assertion of Warren et al. that these types of comparative studies are rare in 
the early intervention (and NI) literature. 


Experimental Design Characteristics and Quality Indicators 


Using quality indicators derived from the What Works Clearinghouse (2010) and Horner et al. 
(2005), none of the 40 SCD studies met all quality indicators at the highest levels included in the 
coding system. There were only 2 studies that provided sufficient information about child partici- 
pants. Both these studies were transition-based teaching studies. Only 1 study provided sufficient 
information about adult intervention agents, and this study was also a transition-based teaching 
study. Although the independent variable(s) was systematically manipulated in all 40 SCD stud- 
ies, fewer than one fourth reported treatment fidelity data for at least 33% of the intervention 
sessions. Data reported in the reviewed studies generally indicated that procedures were being 
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implemented as planned, although most fidelity data were not derived from detailed observations 
or recordings of teaching episodes. Findings in the present review are consistent with previous 
reviews that have examined the reporting of treatment integrity data (Armstrong, Ehrhardt, Cool, 
& Poling, 1997; Wheeler, Baggett, Fox, & Blevins, 2006). Although the importance of collecting 
and reporting treatment fidelity data has been repeatedly emphasized (e.g., Gresham, Gansle, & 
Noell, 1993; LeLaurin & Wolery, 1992; McIntyre, Gresham, DiGennaro, & Reed, 2007), find- 
ings from our review suggest the continuing need for improved data collection and reporting 
practices. 

Serial collection of dependent measures of child performance related to targeted skills 
occurred in all 40 SCD studies, and IOA data were collected for at least 20% of sessions across 
conditions in 32 studies. When behavioral observations were used as dependent measures, the 
metrics used to quantify child performance varied widely. Although different metrics can be 
selected for use within individual studies to quantify functional relationships, the use of different 
child performance metrics across studies makes it difficult to integrate and summarize findings 
without the use of result interpretation aids such as effect-size estimates. Effect-size estimates for 
SCD, however, are less developed than those for group experimental designs, and there are not 
yet uniformly agreed-upon methods or standards for these estimations (Kratochwill et al., 2010). 

The majority of the studies (90%) included at least three attempts to demonstrate an experi- 
mental effect at three different points in time with a single participant or across three different 
participants, and 93% of the studies included at least three data points in each phase. These qual- 
ity indicator findings are particularly important for supporting inferences about functional rela- 
tionships between implementation of NI and child performance and learning. 

None of the three group experimental research studies met all nine quality standards used in 
our review. Methodological strengths of the three studies included adequate description of treat- 
ment and control conditions, reporting reliability for outcome measures and reporting effect sizes 
or sufficient information to calculate effect sizes. All other indicators were met by either one or 
two studies with the exception of providing information about attrition, which was not included 
in any of the studies. 


Child Outcomes 


Regarding child outcomes, findings suggest that NI approaches resulted in acquisition of targeted 
skills for almost all young children with disabilities enrolled in the studies. The exceptions were 
the 2 studies that used a combined approach, with only 70% of the children having positive skill 
acquisition outcomes. The 18 studies reporting generalization data showed some children gener- 
alized skills across different settings, materials, and people, although generalization data were 
less robust for embedded instruction relative to other NI approaches. In the 20 studies that evalu- 
ated maintenance, most children maintained skills over time, although maintenance data were 
less robust for transition-based teaching relative to other NI approaches. 

Findings with respect to skill acquisition are quite promising. Several issues remain to be 
addressed, however, about relationships between implementation of NI approaches and child 
learning outcomes. First, given that primary benefits of NI approaches described in the literature 
include generalization and maintenance of learned skills, the finding that fewer than 50% of 
published studies included generalization and maintenance data was unexpected. Nevertheless, 
these findings are consistent with those of Hepting and Goldstein (1996), who reported that only 
56% of studies included in their systematic literature review focused on naturalistic language 
intervention included generalization data. Second, although 43 studies reported skill acquisition 
data, only 6 specified an acquisition criterion, which limits comparisons about the robustness of 
child outcomes across studies absent calculating effect sizes. Third, the limited comparative data 
related to skill acquisition limit inferences related to the efficiency and effectiveness of NI 
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approaches when compared with other instructional approaches (e.g., direct instruction) or dif- 
ferential treatment intensity. 

Although our review highlights the important work that has established a foundational evi- 
dence base for NI approaches in relation to skill acquisition, additional research is needed to 
evaluate which NI approaches are most effective for which children and under what conditions. 
For example, future research might reveal NI approaches to be relatively less efficient for skill 
acquisition on certain types of learning targets (e.g., preacademic) but relatively more efficient 
when a learning target focuses on a functional communication, social, or adaptive skill that is 
used across people, settings, or materials and over time. 


Delimitations and Limitations 


The present review is delimited by its focus on NI approaches implemented in preschool class- 
rooms during typically occurring activities and with preschool children with or at risk for dis- 
abilities or delays. We did not evaluate studies focused on implementation of these approaches in 
clinic-based settings, in families’ homes, or in isolated locations in the classroom. Space limita- 
tions imposed by journals or the choices made by researchers about which study design features 
or procedural components to include in published work likely affected the summary data reported 
in this review. 


Implications for Future Reporting and for Practice 


Findings from the present review suggest several implications for future reporting practices. 
First, naming and defining the NI approach implemented and reporting the core components or 
active ingredients of the approach are critical for “unpacking” the key procedural components. 
Sufficiently describing these procedural components would allow comparative evaluations 
across NI approaches and improved understanding about relationships between NI implementa- 
tion and child learning outcomes. At a minimum, researchers should describe how the four fea- 
tures that distinguish NI approaches were operationalized in the study. Second, participant 
attributes, treatment fidelity, and treatment intensity data should be reported in sufficient detail to 
permit replication and the examination of functional relationships among study participants, 
intervention implementation, intervention intensity, and child learning outcomes. Third, child 
generalization and maintenance data should be collected and reported, given the emphasis placed 
on these two learning outcomes in the NI literature. Fourth, researchers should gather social 
validity data to inform evaluations about which components of NI approaches practitioners 
believe are feasible, acceptable, and useful to implement in typically occurring activities in pre- 
school classrooms. In the present review, only 5 of 43 studies reported such data. Despite these 
findings, the case for “subjective measurement” remains relevant and important (cf. Wolf, 1978). 

Findings from the present review extend the contributions from a review of naturalistic lan- 
guage interventions conducted by Hepting and Goldstein (1996). First, our review focuses on NI 
approaches, including embedded instruction, described in the empirical literature through 2013 
versus naturalistic Janguage intervention approaches. Second, we examined NI approaches used 
to support child learning across several outcome areas, including communication. At least one 
finding from the present review is consistent with the finding reported by Hepting and Goldstein. 
Across both reviews, it was challenging to identify the specific procedural components of the 
naturalistic language intervention or NI approach implemented. This finding persists in the litera- 
ture despite Rule et al.’s (1998) guidelines related to describing precisely independent and depen- 
dent variables focused on NI. 

Efforts to identify evidence-based practices have provided an impetus for attending to higher 
quality standards in both the conduct and reporting of research, including more transparent ways 
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to judge the rigor and quality of group experimental and SCD studies. These developments might 
motivate researchers to attend to comprehensive sets of quality reporting indicators (cf. Goldstein, 
Lackey, & Schneider, 2014). With adoption of refinements in research methods and reporting 
practices, more explicit documentation related to implementation and outcomes from NI 
approaches should occur. 

The research studies reviewed in this article offer evidence that NI is reliably associated with 
young children with disabilities acquiring targeted skills. As additional evidence for NI approaches 
continues to accumulate, the field will likely need to identify effective and efficient professional 
development strategies for supporting practitioners’ fidelity of implementation of these 
approaches. Practitioners will need to learn about the key features and practices of NI (e.g., what 
to teach, when to teach, how to teach, how to evaluate implementation of instruction and child 
learning outcomes). They will likely benefit from implementation support about how to make 
informed decisions about the children for whom these approaches are most likely to be effective 
and under what conditions instruction should be provided. 
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